To solve the problems of high-dimensionality, giant-computation and redundancy of Gabor features in current Facial Expression Recognition (FER), a new FER algorithm based on Gabor Parameter Matrix (GPM) and improved Adaboost was proposed. Firstly, the GPM was defined by combining pixel information of image and parameters of Gabor wavelet kernel; Secondly, the idea of Genetic Algorithm (GA) was introduced into Adaboost to improve its searching performance, then the improved Adaboost was used to select optimal features corresponding to the elements in GPM to build strong classifiers, thereby the dimensionalities, redundancy and calculation amount of Gabor features could be reduced by feature selection; Finally, on the basis of building several strong classifiers, a multi-expressions classification algorithm was developed to implement FER. The experimental results on Matlab indicate that average expression recognition rate of the proposed algorithm is 89.67%, and the selection efficiency of optimal features is improved significantly.